The data learning problem arises also in cognitive architectures other than Soar

The data learning problem is a phenomenon that arises when an agent employing a cognitive architecture faces the task of acquiring declarative information from an external source, such as the “answer” to a “question”. Because the agent has to pay attention to both question and answer in order to learn the association between them, it is problematic for the agent to learn to produce the answer in response to the question alone. This observation helps shape the basic characteristics of human memory. The problem was first reported with the Soar architecture, but it arises also in ACTR, and this paper argues that it will occur in any cognitive architecture, connectionist as well as symbolic, which is specified in a sufficiently explicit manner to avoid having the theorist act as an implicit homunculus for the agent. The Data Learning Problem The data learning problem is a phenomenon that arises when an agent faces the task of acquiring declarative information provided by an external source. Paradigmatic tasks are, in the laboratory, paired associates, where in response to a stimulus word such as shelf, the agent has to learn to say the digit eight; or in everyday life, learning the correct answer to a question, such as that the capital of France is Paris. In such situations, both the question and the answer (or the “stimulus” and the “response”) must be present and attended to by the agent, in order to learn the connection between them. Thus the agent must process both the question “capital of France?” and the answer “Paris” if it is to acquire the relevant fact. It is therefore problematic for the agent to acquire a directed association from the question to the answer, such that it will be able to retrieve the answer in response to the question alone, without the answer also being present. The data learning problem was first identified in work with the Soar cognitive architecture. However, this paper shows that the problem occurs with ACT-R as well, and argues that it arises in any architecture which is sufficiently theoretically explicit. The Data Learning Problem in Soar The data learning problem was first noticed and discussed (and termed the data chunking problem) by Rosenbloom, Laird & Newell (1987; Newell, 1990) in early work with Soar, and later substantially re-analysed by others (Vera, Lewis & Lerch, 1993; Young & Lewis, 1999). All these authors trace the problem back to the learning mechanism employed by Soar. In Soar, all knowledge — both declarative and procedural — is encoded as if ⇒ then associations in production rules. These production rules implement a search in a state space, by repeatedly identifying an operator to apply to the current state, thereby tranforming it to a new state. Learning can occur only when Soar encounters a situation, an impasse, where it cannot proceed with its normal process of choosing operators and applying them to the current state. In the impasse, Soar sets up a new state, called a substate, and starts a new search whose purpose is find further information that resolves the impasse and allows the original search to continue. Soar learns a new production rule when it finds information that helps the resolution of the impasse and which it associates with the original state. For that new production rule, the ‘then’ part simply consists of the new information. The ‘if’ part consists of the information that existed before the impasse was encountered and to which the new information is connected by the sequence of production rule firings. That description sounds a bit complicated, but the process is quite simple in concrete cases. It also has some surprising consequences. Consider a Soar agent learning about the capital of France. As analysts outside of the process, we can see that the production rule the agent needs to acquire is of the form: Q: capital of France? ⇒ A: Paris (R1) However, in the actual learning situtation, the agent needs to be told both the question and the answer, otherwise it will have no way of knowing what the answer is. So the agent encounters an impasse because it doesn’t know the answer to the question What is the capital of France?, and that impasse is resolved when it is told that the answer is Paris because it can then respond “Paris”. But because both the question and the answer were consulted in giving that response, the production rule actually learned takes the form not of R1 but of: Q: capital of France? & A: Paris⇒ A: Paris (R2) Rule R2 encodes the information “If you are given the question What is the capital of France? and the answer Paris, then respond ‘Paris’”. This is obviously not equivalent to the desired rule R1, because it is applicable only when it is given both the question and the answer. It therefore will not fire when given only the question, which is what R1 does and what the agent needs to be able to do if it is to know how to answer the question. How, then, can the agent ever acquire rule R1? The solution to this problem (Rosenbloom et al, 1987) requires us to note that although rule R2 cannot serve to recall the answer to the question, it can serve as an episodic recognition rule encoding the fact that the question and answer have been encountered together. Rosenbloom et al point out that if the agent is able to generate plausible candidates for the answer, then the recognition rule R2 can fire to pick out which one was previously presented as the answer. In this case the agent asked What is the capital of France? might generate and consider well-known cities in France, such as Calais, or Lyon, or Marseilles, or Paris. For the first three nothing would happen, but for the fourth, rule R2 would fire, identifying Paris as the answer and enabling the agent to respond. Because this time the answer is generated internally, the impasse occurs when only the question is present, not the answer, and so when Paris is given as the response the new rule learned is R1. Thereafter the agent can answer the question by recalling the answer, i.e. by firing rule R1, and the generation process does not have to be repeated. Young & Lewis (1999) re-cast the process as a choice between four methods of answering the question: 1. If there is a production rule that provides the answer to the question, then it fires to deliver the answer. 2. If the answer already exists in working memory, then it gets picked up and delivered as the answer. 3. If possible candidates for the answer can be generated, and if a production rule exists that will recognise the answer, then candidates are generated until the answer is recognised. 4. The answer is sought from external sources. These methods are listed in order of increasing effort. It is therefore rational for the agent to attempt the methods in that order. On the first pass, when the agent is first asked What is the capital of France?, none of the first three methods is applicable: the agent does not yet have a rule like R1; the answer is not available in working memory; and although the agent can try generating plausible candidates, there is no basis for recognising the correct answer. So the agent needs to be told (method 4) that the answer is Paris. It then has the answer in working memory, and so can respond (using method 2). The result is the acquisition of rule R2, the recognition rule. On the second pass, when the agent is asked the question again, once more neither method 1 nor method 2 is applicable. Method 3 however does now work. As just described, the agent proceeds to generate plausible candidates, and when Paris is generated rule R2 fires, identifying it as the correct answer. In consequence, the recall rule R1 is acquired. On the third and subsequent passes, once the agent is asked the question rule R1 fires (method 1) to give the answer. The agent has now clearly learned the geographical fact. In their discussion, Young & Lewis (1999) stress several characteristics of this process: • Learning the fact, i.e. acquiring the recall rule R1, requires the agent at one stage to generate the answer from its own internal, cognitive resources (Newell, 1990). This observation connects with wider discussion of the “reconstructive” nature of human memory (e.g., Bahrick, 1970). • The sequence of learning emerges as a by-product of the agent’s rational attempts to answer the question. At no point is there a deliberate attempt to learn, and there is no appeal to any mechanism for “selfprogramming”, i.e. deliberate learning. • The learning process is inherently multi-pass, not because of any kind of gradual strengthening but because of the inherent informational dependencies in the process. The first pass is needed to acquire the recognition rule, which is used in the second pass to acquire the recall rule. Only in the third and later passes can recall occur. The learning process just sketched arises as a consequence of the architecture, and is lean in its theoretical assumptions. Young & Lewis see these as theoretically desirable properties of any explanation of learning. In particular, the analysis argues against the need for any homunculus-like mechanism for deciding what should and what should not get learned. Generality of the Data Learning Problem Although the data learning problem has to date been documented and analysed only for Soar, our account of its origins suggests that the problem is potentially of wide generality and should arise also for other, or perhaps even all, cognitive architectures. After all, the only assumption needed to carry the argument is that learned memory associations incorporate the information presented — and that would seem to be a reasonable tenet of most architectures and associative theories. So, is the data learning problem just a quirk of Soar, or is it instead pointing us to a deep truth about cognitive architectures in general? If we strip away the particularities of Soar, the bare bones of the argument look something like this: • We assume that memory (or at least, part of memory) is encoded in the form of directed associations, cue ⇒